Skip to content

feat(pools): add opt-in reset-aware account pool scheduling - #55

Merged
ALikesToCode merged 1 commit into
integrate/gateway-waves-1-4from
feat/pool-reset-scheduling
Oct 9, 2026
Merged

ALikesToCode merged 1 commit into
integrate/gateway-waves-1-4from
feat/pool-reset-scheduling

Conversation

@ALikesToCode

Copy link
Copy Markdown
Owner

Summary

Adds opt-in reset-aware account pool scheduling. With POOL_RESET_SCHEDULING_ENABLED off (the default), account selection, cooldowns, usage snapshots and errors are unchanged. Tests compare whole selection sequences with the flag on and off.

  • Observations:
    • Taken only from data the gateway already receives: authorized usage snapshots it already fetches, and metadata on classified request results. A quota 429 with retry_after_seconds counts as an exhausted window with a relative reset.
    • NanoGPT's subscription usage endpoint is excluded.
    • No probes, refresh calls, browser sessions, rewards or check-ins.
  • Choice:
    • Applies within the pool and priority tier the gateway already picks from.
    • Freshly exhausted accounts are skipped. The earliest fresh reset goes first, with the configured order breaking ties. Accounts without fresh evidence keep their positions.
    • Stale, missing or contradictory observations fall back to the existing order.
    • Prompt cache affinity can only prefer an account that is still eligible.
  • All exhausted: when every account is cooling or freshly exhausted, the existing model-cooldown 429 is raised before dispatch. Its body is unchanged, and Retry-After comes from the soonest fresh reset, clamped to 1–3,600 s.
  • NanoGPT: key validation probes keep their existing cadence. The scheduler only filters and orders the keys before that logic runs. A validated active key stays in use until its usual re-check, unless a fresh observation shows it exhausted.
  • Bounds:
    • POOL_OBSERVATION_TTL_SECONDS defaults to 60 and accepts 5–3,600.
    • At most 10,000 observations and 32 windows each, in process only.
    • Only salted HMAC credential identities are stored, never keys or prefixes. Nothing is logged.

Details are in docs/pool-reset-scheduling.md.

Review rounds

Round 1 skipped NanoGPT's existing validation probes whenever the feature was on. That removed a safeguard against revoked keys. Round 2 restores them unchanged, and tests prove the probe sequence matches flag-off when there are no observations.

Shared registrations

POOL_RESET_SCHEDULING_ENABLED and POOL_OBSERVATION_TTL_SECONDS are added to the Container env keys. There are no routes, Worker modules, tables or headers.

Left for the Wave 5 integration

Optional: pass parsed usage windows from responses into the pool result calls (usage_windows):

  • proxy_service.make_request → CredentialPool.record_headers;
  • routes/proxy.py → NanoGPTKeyPool.record_result;
  • intelligence_transport → NanoGPTUnifiedKeyPool.record_result.

Until then, observations come from usage snapshots and classified quota 429s.

Tests

  • tests/test_pool_reset_scheduling.py covers:
    • fresh vs stale observations;
    • exhausted accounts near their reset staying excluded;
    • stable ties;
    • account and model scopes;
    • clock skew bounds;
    • contradictory quota;
    • affinity not overriding health;
    • zero refresh or generation calls;
    • unchanged NanoGPT probe sequences;
    • 429 without probes when everything is exhausted.
  • Full Python suite: 3,965 passed, 2 skipped. ruff, mypy and bandit are clean.
  • npm run test:worker: 1,460 passed. npm run test:integration: 7 passed.
  • test:deploy, free routes, client headers and the wrangler deploy --dry-run bundle all pass.

CI notes

  • GitHub python/node jobs: the dependency-audit steps fail on every branch, including main. fix(deps): clear the CI dependency audit advisories #42 fixes that.
  • Workers Builds: fails on this stack because the Wave 3 base adds the AdmissionCoordinator Durable Object migration, which wrangler versions upload cannot apply. This is expected until wrangler deploy runs on main.

This is part of Wave 5. It is based on integrate/gateway-waves-1-4, the Wave 4 integration branch. The README line comes with the Wave 5 integration PR.

Behind POOL_RESET_SCHEDULING_ENABLED, account pools use fresh reset observations from usage snapshots and response metadata the gateway already receives. Within the existing pool and priority tier, freshly exhausted accounts are skipped and the earliest fresh reset goes first; stale, missing or contradictory observations keep the existing order. When every account is cooling or exhausted, the existing 429 carries a Retry-After from the soonest fresh reset instead of sending another request. NanoGPT key validation keeps its existing cadence, and the scheduler adds no probes. Observations live in process for POOL_OBSERVATION_TTL_SECONDS and store only HMAC credential identities.
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
❌ Deployment failed
View logs
multillm-proxy 2e27ecc Oct 09 2026, 02:09 AM

@coderabbitai

coderabbitai Bot commented Oct 9, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 10ba53f4-1a2c-4711-a89e-ad20775c0282

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ALikesToCode
ALikesToCode merged commit c2d897a into integrate/gateway-waves-1-4 Oct 9, 2026
4 of 7 checks passed
@ALikesToCode
ALikesToCode deleted the feat/pool-reset-scheduling branch October 26, 2026 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant